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ABSTRACT 

The purpose of this study was to investigate two 
questions pertaining to student evaluation of teachers. These are: 

(1) Do students indeed provide the instructor with information about 
instructional practices that he doesn’t already know? and (2) If this 
is the case, to what extent is it true at a variety of colleges and 
for a significant proportion of instructors? The study was conducted 
by comparing student ratings or descriptions of instruction with the 
teachers* own self-reported descriptions. The correlation between the 
2 sets of descriptions or ratings were not particularly high, 
indicating only modest agreement in the way faculty and students 
perceived instruction. In particular, instructors and students did 
not agree on Ciie extent to which students are free to ask questions 
or give opinions in class, on the extent to which instructors are 
concerned with student learning, on the amount of agreement between 
objectives and what is being taught, on instructor openness to other 
viewpoints, on the extent to which instructors inform students of how 
they would be evaluated, on whether the instructor encourages 
students to think for themselves, and on the clarity of course 
objectives. These discripancies between the 2 sets of ratings not 
only underscore the need for student feedback, but also suggest 
specific areas of instruction where feedback is most essential. 
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WHAT INSTRUCTORS LEARN FROM STUDENTS: 

STUDENT AND FACULTY RATINGS OF INSTRUCTION 1 

John A. Centra 
Educational Testing Service 

Introduction 

The extensive use of student ratings of instruction at the college level 
over the past several decades has had as its primary purpose the improvement 
of instruction. Student evaluations typically are seen only by the instructor 
and are intended to help improve their teaching. Underlying this use are 
several assumptions, among which is that instructors are learning something 
new from students about their teaching. 

Do students indeed provide the instructor with information about instruc- 
tional practices that he doesn't already know? And if this is the case, to 
what extent is it true at a variety of colleges and for a significant propor- 
tion of instructors? 

The purpose of this study \was to investigate these questions by comparing 
student ratings or descriptions of instruction with the teacher's own self- 
reported descriptions. Discrepancies between the two sets of ratings, if 
found, would not only underscore the need for student feedback but would also 
suggest specific areas of instruction where the feedback is most essestial. 

In addition, self-ratings have been suggested by some as a worthwhile mode 
of instructional evaluation. More knowledge about their relationship with stu- 
dent ratings could provide a better understanding of self -ratings, and possibly 
lead to ways in which they could serve as another source of classroom evaluation. 
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Procedure 

The sample for the study consisted of 31+3 teaching faculty at five 

institutions of higher education. The five institutions included two state 

colleges, one of which had a predominantly black enrollment, a selective 

liberal arts college, a multipurpose college, and an urban community college. 

None of these institutions had, at the time of the study, a systematic program 

to collect student ratings, nor did a significant portion of their faculty 

collect student ratings on their own. The majority of teachers in this study, 

therefore, were not familiar with how students might rate their instruction. 

Teachers were randomly selected to participate in the study, and between 

75-90$ of those selected from each college participated. 

A 21-item instructional report questionnaire was used in the study. 

Included were items that faculty members in an earlier study had identified 

as providing information they would like to receive from students (Centra, 

2 • 

1972). Among the dimensions of instruction included were the organization 
of the course, student- teacher interaction, instructor communication, student 
effort, and stimulation of students. Previous factor analytic studies had 
identified several of these as dimensions that effectively differentiated 
among instructors (Isaacson, KcKeachie, et al., 196hj Hodgson, 1958 j Gibb, 

1955 i Coffman, 195U). 

Responses to 17 of the items were on a four point agree -disagree scale, 
with a not applicable option also «, provided. The four remaining items used 
a four or five point scale with different response options for each item. 



The form actually contained 23 items, 21 of which could be reworded for 
reasonable instructor self-ratings. 
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The wording for each of the statements in the questionnaire differed slightly 
for students and instructors. For example, an item on course objectives was 
worded as follows for each group: 

Students: The instructor's objectives have been made clear 

Teachers: I feel my objectives for the course have been made clear to 

students 

Teachers were asked to "describe this course, your teaching, or the stu- 
dents enrolled. 11 They wens told that the reason for obtaining this self-report 
was to see which items were tapping information already known to most instructors 

The data were collected at mid-semester of the fall 1971 term. Instructors 
administered the rating form in one class of their own choosing, with the under- 
standing that only they would receive a summary of their students' responses. 

Faculty- student comparisons were made primarily in two ways. First, for 
each item the instructors' self -descriptions were correlated with the mean 
responses of students in their class (N=3U3) • In this way the relationship 
between the two sets of ratings could be noted. Second, the differences between 
the way faculty as a group and students as a group rated or described instruction 
were investigated by a comparison of means. For this analysis responses for all 
faculty were compared to the average of the student class means on each item. 



Results and Discussion f~ * 

The results of the comparison of means and the correlational analysis for 
items 5-21 are presented in Table 1. The correlation between the two sets of 
descriptions or ratings were no.t particularly high, indicating only modest 
agreement in the way faculty and students perceived instruction. While most of 
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Faculty-Student Comparisons to Instructional Report Questionnaire 
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the items were statistically significant due to the large N (3h3 ) > the median 
correlation was only .21. 

Also listed in Table 1 are the mean faculty responses for the items, the 
mean of the student means, the results of the T tests, and the number of col- 
leges where the difference between the means was significant. Responses for 
items 5-21 could range from one for "strongly agree", to four for "strongly 
disagree j" thus, lower values represent greater agreement with each statement. 

The comparisons of the mean values indicate that instructors as a group generally 
rated or described their teaching more favorably than did their students. In 
particular instructors and students did not agree on the extent to which students 
are free to ask questions or give opinions in class (item lU) , on the extent to 
which instructors are concerned with student learning (ll), on the amount of 
agreement between objectives and what is being taught. (6), on instructor open- 
ness to other viewpoints (20), on the extent to which instructors inform stu- 
dents of how they would be evaluated (l6), on whether the instructor encourages 
students to think for themselves (10), and on the clarity of course objectives 
(5). For each of these seven items, instructor-student differences were notable 
at either four or all five of the colleges. 

On the other hand there was little difference between the faculty and stu- 
dent groups in their ratings of the instructor preparation for class (l5) and 
on the extent to which course objectives were being accomplished (21) . For the 
remaining eight items, the differences were modest and in many instances not 
significant within a college. 

But probably more important than a simple comparison of the way an average 
instructor and an average class rated, instruction is some knowledge of how many 
instructors saw themselves far differently than their students did. A distribu-. 
tion of the differences between each instructor's responses and those of his - 
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class (i.e., the class means) provides that information. Presented in Table 2 
is a summary of the results of such a distribution. For each item, the percent- 
age of instructors who gave themselves "considerably poorer" or "considerably 
better" ratings is indicated within each college and for the total sample. A 
difference of .63 or greater was used to define "considerably poorer or better" 
because a difference of at least that great would appear to be large enough to 
have some practical significance ; it is also the approximate standard deviation 
for most of the student item responses. 

For most of the items, between a fourth and a third of the instructors 
described or rated themselves considerably better than their students did. 

The median, in fact, was. just under 30 percent for all 3^3 instructors and 
their classes. Forty-one percent of the instructors gave themselves better 
ratings on item lU: students are free to ask questions or give opinions in 
class; and 36 percent on item 11: the instructor is concerned about whether 
students learn and tries to be actively helpful. Both items deal with faculty- 
student interaction as do items 8, 10, and 16 for which fairly high percent- 

ages of instructors also gave themselves better ratings. The faculty- student 
interaction dimension, then, appears to be one on which, a sizable number of 
instructors and their students do not agree, and on which student reactions 
would appear to be especially crucial. Other similar areas would be the 
instructor's openness to other viewpoints (item 20) and the agreement between 
announced objectives for the course and what was being taught (6). 

A surprisingly large percentage olLihstructors rated themselves poorer 
than students did in a few areas. Fifteen percent rated themselves more 
poorly on class preparation and 12 percent were less satisfied that they were 
accomplishing course objectives.. In general, however, only between four to 
eight percent of the teachers gave themselves considerably poorer ratings. 







Results of the Distribution of Differences between Faculty- Student Responses 
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One of the items in the form was unique in that it elicited opinions on 
student effort in the course (19). For students, the exact wording was: I 

have been putting a good deal of effort into this course; for instructors it 
was worded: Students seem to be putting a good deal of effort into this course. 
The results for this item, as one might expect* were much different than those 
for other items. Compared to students' responses, 18 percent of the faculty 
thought students generally were putting considerably less effort into the 
course, while 10 percent gave students better ratings on effort than students 
gave themselves. In other words, in this instance students have tended to give 
themselves better ratings just as instructors did on so many of the previous 
items. 

An inspection of the differences within each college indicates fairly 
similar results with the exception of college five. In comparison to the 
other four colleges, higher percentages of the instructors at college five 
rated themselves considerably better than did their students on a majority of 
the items. While it is not possible to conclude much on the basis of one 
college, it is interesting to note that college five was the smallest and most 
selective of the colleges in the study. Moreover, instructors at college five 
were given the poorest student ratings among the five colleges, whereas their 
self-ratings were not much different or poorer than those of instructors 
elsewhere. Thus the gap between instructor- student ratings at college five 
was due to the poorer ratings by students, perhaps because of higher expecta- 
tions on their part, rather than on better ratings by instructors. 

Presented in Table 3 is a summary of responses to the first four items, 
whicK used varied responses rather than agree-disagree options. The items deal 
with the pace, the level of difficulty, and the work. load of the course, as well 
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Table 3 



Faculty-Student Comparisons at Five Colleges and Total (N = 343 ) , 
For Four Items in Instructional Report Questionnaire 



Percentage Responding 



Students 



Faculty 



1 Pace at which material College 





is covered: 


1 


2 


3 


4 


5 


Total 


1 


2 


3 


4 


C 

> 


Total 




Very or somewhat slow 


9 


10 


7 


8 


6 


9 


22 


24 


10 


8 


14 


16 




Very or somwhat fast 


26 


20 


27 


23 


33 


25 


20 


28 


24 


30 


30 


27 


2 


Level of difficulty of 




























course for students • 




























enrolled: 




























Very or somewhat elementary 


11 


13 


10 


10 


9 


11 


10 


7 


10 


4 


8 


7 




Very or somewhat difficult 


31 


25 


32 


21 


38 


30 


26 


31 


37 


37 


41 


34. 


3 


Work load of course rela- 




























tive to others: 




























Lighter 


18 


22 


17 


19 


18 


19 


25 


24 


21 


17 


14 


20 




Heavier 


20 


21 


27 


29 


27 


25 


35 


23 


32 


32 


33 


30 



4 Extent to which examples 
and illustrations were 
used: 



Frequently 


60 


70 


76 


67 


58 


67 


88 


75 


86 


82 


65 


80 


Occasionally 


28 


26 


20 


26 


34 


26 


12 


21 


14 


18 


32 


19 


Seldom 


10 


4 


4 


6 


8 


6 


0 


2 


0 


0 


3 
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Never 


2 


.1 


1 


1 


1 


1 
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ERIC 
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► 



; as the extent to which the instructor used examples and illustrations. Once 

! again there were student- instructor differences although they were not particu- 

larly large. Instructors tended to think they more often used examples and 

illustrations, and at three of the colleges instructors more likely considered 
fc* 

the pace at which material was covered to be slow. College five, the selective 
liberal arts college, was once again noteworthy in that its faculty and to some 
! extent the students reported less frequent use of examples or illustrations in 

courses. 

■ 

Summary and Conclusions 




A comparison of students' ratings of instruction with teachers' self- 
reported ratings in over 300 classes at five colleges disclosed a modest 
relationship between the two set of evaluations. The median correlation for 
17 items was .21, indicating that faculty members generally evaluate or describe 
their teaching quite differently from the way it is evaluated or described by 
students. Not surprisingly, the highest correlations occurred for the more 
factual items, on which there was somewhat less chance for disagreement (e.g., 
the instructor informs students of how they would be evaluated), while items 
eliciting opinions (e.g., the instructor is using class time well) resulted 
in the lowest correlations. 

Clark and Blackburn (1971) recently reported a similarly low to moderate 
correlation of .19 be tween students and faculty self-ratings on a single over- 
all measure of teaching effectiveness. Thus, whether instructors give themselves 
an overall rating of teaching effectiveness, or rate more specific instructional 
practices, they apparently do not agree with evaluations being made by students.. 
In fact, overall self-ratings and colleague ratings, according to Clark and 

Blackburn's data, correlated only slightly better (r = .28). 

10 
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In addition to the general lack of agreement between self and student 
evaluations, there was also a tendency for teachers as a group to give them- 
selves better ratings than students gave them. This discrepancy, however, 
was most notable for between a fourth to a third of the 3h3 instructors in 
the study, and in particular for items related to student-instructor inter- 
action, course objectives and the instructor's openness to other viewpoints . 
These areas of instruction, then, would seem to be particular ones in which 
a sizable proportion of teachers could profit from student feedback. 

In conclusion, the results of this study would appear to argue for the 
collection of student ratings as a means of providing many instructors with 
information they do not already have about their teaching. Whether instructors 
actually use this new information is, of course, another question. 
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